Papers with confidence estimator
A Confidence-based Partial Label Learning Model for Crowd-Annotated Named Entity Recognition (2023.findings-acl)
Copied to clipboard
Limao Xiong, Jie Zhou, Qunxi Zhu, Xiao Wang, Yuanbin Wu, Qi Zhang, Tao Gui, Xuanjing Huang, Jin Ma, Ying Shan
| Challenge: | Existing models for named entity recognition (NER) are based on large-scale labeled datasets, which always obtain using crowdsourcing. |
| Approach: | They propose a CONfidence-based partial Label Learning method to integrate prior and posterior confidences for crowd-annotated named entity recognition models. |
| Outcome: | The proposed model improves on real-world and synthetic datasets compared with baselines. |
The Craft of Selective Prediction: Towards Reliable Case Outcome Classification - An Empirical Study on European Court of Human Rights Cases (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing COC tasks prioritize high task performance over model reliability . however, large models exhibit overconfidence and Monte Carlo dropout methods produce reliable confidence estimates . |
| Approach: | They conduct an empirical investigation into how various design choices affect the reliability of COC models within the framework of selective prediction. |
| Outcome: | The proposed model is able to predict the outcome of a legal case based on the text of the case facts and is compared with other models using a pre-training corpus. |
Selective Temporal Knowledge Graph Reasoning (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing models cannot abstain from uncertain predictions, which will bring risks in real-world applications. |
| Approach: | They propose to abstain from uncertain future facts by using a confidence estimator . they take both the certainty of the current prediction and the accuracy of historical predictions into account . |
| Outcome: | The proposed abstention mechanism helps existing models make selective predictions instead of indiscriminate ones. |